<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
	"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
    <title>uSTL - the small STL library</title>
    <link rel="stylesheet" type="text/css" href="default.css" />
    <link rel="icon" type="image/png" href="favicon.png" />
    <meta http-equiv="Content-Type" content="text/xhtml+xml; charset=ISO-8859-1" />
    <meta name="viewport" content="width=device-width, initial-scale=1" />
    <meta name="Description" content="API and usage description for uSTL, a size-optimized STL implementation" />
    <meta name="Keywords" content="C++, STL, template, bloat, optimization" />
</head>
<body>
<div id="pagectr">
<div id="header">
    <h1>uSTL</h1>
    <h6>Candy for the optimization nut in you</h6>
    <hr/>
</div>
<div id="content">

<h2 id="introduction">Introduction</h2>
<p>
This project is <strong>obsolete</strong>. When uSTL was created
back in 2003, the C++ standard library implementation was implemented
poorly, without any concern for code size, template bloat, or memory
footprint. This library was then intended to remedy those problems
and succeeded reasonably well, providing savings of several kb of code
size per container, several megabytes of savings in memory footprint,
and a variety of small space-saving and performance improvements. Today
the official standard library implementation has improved drastically,
owing to improvements in the C++ standard itself, as well as laudable
optimization efforts by gcc developers. Furthermore, due to proliferation
of C++-using applications, memory footprint reductions from using
uSTL are no longer possible. Compared to the official standard library
implementation, uSTL provides very little advantage, and many drawbacks
due to the design tradeoffs and limitations that made the space savings
originally possible. Therefore I strongly recommend that uSTL no longer
be used in any new projects. I shall continue to maintain the library
for existing users, if any, but would likewise recommend that you too
use the official standard library instead.
</p>
<h2>Contents</h2>
<ul>
    <li><a href="#Installation">Installation</a></li>
    <li><a href="#Containers">Containers and Iterators</a></li>
    <li><a href="#Strings">Strings</a></li>
    <li><a href="#Algorithms">Algorithms</a></li>
    <li><a href="#Memblocks">Memblock and Memlink</a></li>
    <li><a href="#Streams">Streams</a></li>
    <li><a href="#Arrays">Arrays</a></li>
    <li><a href="#Exceptions">Exceptions</a></li>
    <li><a href="#Contact">Bug reporting</a></li>
</ul>
<h2 id="Installation">Installation</h2>
<p>
The only dependency is a C++ compiler, gcc 3.4 or clang 3.2. C++14
support requires gcc 5 or clang 3.6. clang does not optimize for size
as well as gcc, producing binaries larger by about 15%, so its use is
not recommended.
</p><p>
The latest version of uSTL can always be downloaded from its GitHub
<a href="https://github.com/msharov/ustl/releases/latest">release page</a>.
If you like living dangerously, you can pull the working branch directly from
<a href="https://github.com/msharov/ustl">git@github.com:msharov/ustl.git</a>.
The mainline source should build on any unix-based system, including
Linux, BSD, MacOS, SunOS, and Solaris. Windows-based systems and embedded
platforms, are not, and will not be supported by the mainline. Unpack and:
</p><pre>
./configure &amp;&amp; make install
</pre><p>
<kbd>./configure --help</kbd> lists available build options.
You might want to specify a different installation prefix with
<kbd>--prefix=/usr</kbd>; the default destination is /usr/local.
Developers will want to build with <kbd>--with-debug</kbd> to get a lot
of assert error checking, which I highly recommend. If you have gcc 4.4
or later, you may want to also use <kbd>--force-inline</kbd>. Unless you
are compiling a package for distribution, <kbd>--with-native</kbd> should
be enabled to tune the output for your processor. If you are the type
to edit configuration manually, it's in Config.mk and config.h. When
it's built, you can run the included tests with <kbd>make check</kbd>.
Finally, here's a simple hello world application:
</p><pre>
#include &lt;ustl.h&gt;
using namespace ustl;

int main (void)
{
    cout &lt;&lt; "Hello world!" &lt;&lt; endl;
    return EXIT_SUCCESS;
}
</pre><p>
If you have at least gcc 3.4, uSTL is built as a standalone library,
without linking to libstdc++ (except on BSD platforms where gcc does not
support it) Because g++ links to it by default, you'll need to link your
applications with gcc, or to pass <kbd>-nodefaultlibs -lgcc_s -lc</kbd>
to g++ if you want to use uSTL to completely replace libstdc++. This is
where most of the space savings happen.
</p><p>
If you build uSTL as a static library, you will also need to explicitly
link all projects using it with -lsupc++ and other libraries that shared
uSTL would have been linked to. The easiest way to get the list is to use
<kbd>pkg-config --libs --static ustl</kbd>. The pkg-config description
file for uSTL is installed if you have pkg-config on your system. The
list is also available in the <tt>STAL_LIBS</tt> variable in Config.mk.
</p>
<h2 id="Containers">Containers and Iterators</h2>
<p>
STL containers provide a generic abstraction to arrays, linked lists,
and other methods of memory allocation. They offer the advantages
of type-safety, the peace of mind that comes from never having to
malloc anything again, and a standard access API called iterators. Each
container's API is equivalent to that of a simple array, with iterators
being the equivalent of pointers into the array. The uniform access
API allows creation of standardized algorithms, discussed futher down,
that work on any container. Here are some examples of using vector,
the container representing a simple array:
</p><pre>
vector&lt;int&gt; v;
v.push_back (1);
v.emplace_back (2);
v[1] = 0;
v.erase (v.begin() + 1);
v.pop_back();
v.insert (v.begin(), 4);
v.resize (15);
</pre><p>
As you can see, a vector is basically the same thing as the arrays
you use now, except that it is resizable. The function names
ought to be self-explanatory with the exception of the addressing
arguments. You can do index addressing and get free bounds checking
with asserts. Incidentally, I highly recommend you work with a debug
build when writing code; uSTL is chock full of various asserts checking
for error conditions. In the optimized build, most such errors will be
silently ignored where possible and will cause crashes where not. That
is so because they are programmer errors, existing because you have a
bug in your code, not because the user did something wrong, or because
of some system failure. Programmer errors assert. User or system errors
throw exceptions.
</p><p>
Vectors are addressed with iterators, which are just like pointers (and
usually are). Calling begin() gives you the pointer to the first element,
calling end() gives you the pointer to the end of the last element. No,
not the last element, the end of it, or, more accurately, the end of the
array. It's that way so you can keep incrementing an iterator until it
is equal to the end() value, at which point you know you have processed
all the elements in the list. This brings me to demonstrate how you
ought to do that:
</p><pre>
foreach (auto, i, v)
    if (*i &lt; 5 || *i &gt; 10)
	*i = 99;
</pre><p>
Although the foreach macro is a uSTL-only extension, it is a one-liner
you can easily copy out of uutility.h if you ever want to switch back to
regular STL.  It is a great way to ensure you don't forget to increment
the counter or run past the end of the vector. The only catch to be aware
of, when inside an iterator loop, is that if you modify the container,
by adding or removing entries, you have to update the iterator, since the
container memory storage may have moved when resized. So, for example,
if you wish to remove certain types of elements, you'd need to do use
an index loop or something like:
</p><pre>
foreach (auto, i, employees)
    if (i-&gt;m_Salary &gt; 50000 || i-&gt;m_Performance &lt; 100)
	--(i = employees.erase (i));
</pre><p>
This is pretty much all there is to say about containers. Create them,
use them, resize them, that's what they are for. There are other
container types, but you will probably not use them much. There's
<code>set</code>, which is a perpetually sorted vector, useful when you
want to binary_search a large collection. There's <code>map</code> which
is an associative container where you can look up entries by key. Its
utility goes down drastically when you have complex objects that need to
be searched with more than one parameter, in which cast you are better
off with vector and foreach.
</p><p>
It is important to mention here that all uSTL containers are built on top
of vector, and have the same iterator invalidation semantics. The standard
library implementation of non-vector containers uses linked backends,
where, for example, inserting into the middle does not invalidate
iterators to other elements. If your code depends on this behavior,
you can not use uSTL. Linked container backends are very inefficient
today because of their poor cache locality, which is the most important
factor for performance on modern processors.
</p>
<h2 id="Strings">Strings</h2>
<p>
Every program uses strings, and STL was kind enough to provide a
specification. uSTL deviates a bit from the standard by not implementing
wchar strings. There is only one <code>string</code> class, which assumes
all your strings will be UTF8-encoded, and provides some additional
functionality to make working with those easier. I did that for the same
reason I dropped the locale classes; bloat. It is simply too expensive to
implement the standard locale classes, as the enormous size of libstdc++
illustrates. If you need them, you can still include them from libstdc++,
but it may be just as simple to use the locale support provided by libc
through printf, which may be called through <code>format</code> functions
in string and ostringstream.
</p><p>
Anyway, back to strings. You can think of the string object as a
char vector with some additional operations built-in, like searching,
concatenation, etc.
</p><pre>
string s ("Hello");
s += ' ';
s += "world?";
s.replace (s.find ('?'), 1, "!");
s[3] = s[s.find_first_of("lxy")];
s[s.rfind('w')] = 'W';
s.format ("A long %zd number of 0x%08lX\n", 345u, 0x12345);
cout &lt;&lt; s &lt;&lt; endl;
</pre><p>
A nonstandard behaviour you may encounter is from linked strings created
by the string constructor when given a null-terminated const string. In
the above example, the constructor links when given a const string and
stays as a const link until the space is added. If you try to write to it,
you'll get an assert telling you to use copy_link first to convert the
link into a copy. Resizing the linked object automatically does that for
you, so most of the time it is transparent. You may also encounter another
instance of this if you try getting iterators from such an object. The
compiler uses the non-const accessors by default for local objects,
so you may need to declare it as a const string if you don't wish to
copy_link. Why does uSTL string link instead of copying?  To save space
and time. All those strings are already in memory, so why waste heap
space and processor time to copy them if you just want to read them? I
thought it a good tradeoff, considering that it is trasparent for the
most common uses.
</p><p>
Other nonstandard extensions include a <code>format</code> function to
give you the functionality of sprintf for string objects. Another is
the UTF8 stuff. Differing a bit from the standard, <code>size</code>
returns the string length in bytes, <code>length</code> in characters.
You can iterate by characters instead of bytes with a special utf8
iterator:
</p><pre>
for (auto i = s.utf8_begin(); i &lt; s.utf8_end(); ++i)
    DrawChar (*i);
</pre><p>
or just copy all the chars into an array and iterate over that:
</p><pre>
vector&lt;wchar_t&gt; result (s.length());
copy (s.utf8_begin(), s.utf8_end(), result.begin());
</pre><p>
To write wide characters to the string, wchar_t values can be directly
given to push_back, insert, append, or assign, in the same way as the
char ones.
</p><p>
A few words must be said regarding reading wide characters. The shortest
possible rule to follow is "don't!" I have received a few complaints about
the fact that all offsets given to and returned by string functions are
byte offsets and not character offsets. The problem with modifying or even
looking for specific wide characters is that you are not supposed to know
what they are. Your strings will be localized into many languages and it
is impossible for you to know how the translation will be accomplished.
As a result, whenever you are hardcoding a specific character value,
or a specific character length (like a three-character extension),
you are effectively hardcoding yourself into a locale. The only valid
operation on localized strings is parsing it via standard delimiters,
treating anything between those delimiters as opaque blocks. For this
reason, whenever you think you need to do something at a particular
character offset, you should recognize it as a mistake and find the
offset by the content that is supposed to be there.
</p><p>
If this philosophy is consistently followed, it becomes clear that
actual character boundaries are entirely irrelevant. There are only
two exceptions to this: first occurs if you are writing a text editor
and want to insert user data at a character position, the second occurs
if you are writing a font renderer and want to translate characters to
glyphs. In both cases you should make use of the utf8_iterator to find
character boundaries and values. Given that these two cases apply to
just a handful of people who are involved in implementing user interface
frameworks, I believe that the opacity restriction is well justified by
the amount of code space it saves for the vast majority of library users.
</p>
<h2 id="Algorithms">Algorithms</h2>
<p>
Algorithms are the other half of STL. They are simply templated common
tasks that take iterator arguments, and as a result, work with any
container. Most will take an iterator range, like (v.begin(), v.end()),
but you can, of course operate on a subset of a container by giving a
different one. Because the usual operation is to use the whole container,
uSTL provides versions of most algorithms that take container arguments
instead of the iterator range.  Here are the algorithms you will actually
find useful:
</p><pre>
copy (v1, v2.begin());		// Copies vector v1 to vector v2.
fill (v, 5);			// Fills v with fives.
copy_n (v1, 5, v2.begin());	// Copies first five elements only.
fill_n (v.begin() + 5, 10, 5);	// Fills elements 5-15 with fives.
sort (v);			// Sorts v.
find (v, 14);			// Finds 14 in v, returning its iterator.
binary_search (v, 13);		// Looks up 13 with binary search in a sorted vector.
lower_bound (v, 13);		// Returns the iterator to where you want to insert 13.
iota (v.begin(), v.end(), 0);	// Puts 0,1,2,3,4,... into v.
reverse (v);			// Reverses all the elements in v.
</pre><p>
The rest you can discover for yourself. There are obscure mathematical
operations, like inner_product, set operations, heap operations, and
many predicate algorithms, taking a function. Predicate algorithms used
to be useless before c++11 added support for lambda functions. With c++11
you can use these generic algorithms much easier:
</p><pre>
int minValue = 5, maxValue = 10, badBalue = 99;
for_each (v.begin(), v.end(), [=](int&amp; v) {
    if (v &lt; _minValue || v &gt; _maxValue)
	v = _badValue;
});
</pre><p>
And yes, it really does work. Doesn't always generate much bloat either,
since the compiler can often see right through all this trickery and
expand the for_each into a loop without actually creating the functor
object. However, the compiler has a much harder time when you start
using containers of complex objects or operating on member variables
and member functions. And in general, functional programming should
be discouraged. The equivalent imperative version is usually easier
to read and it represents the way the processor really works, as well
as the way you should be thinking about code. Leave functional to the
academics and the overly mathematical. In the real world, we do things
step by step.
</p>
<h2 id="Memblocks">Memblocks and Memlinks</h2>
<p>
The STL specification is only about containers and algorithms, the stuff
described from here on is totally non-standard, so by using them you'll
have to stick with uSTL as your STL implementation. I think it's worth
it, but, of course, the choice is up to you.
</p><p>
The major difference between the standart STL implementation and uSTL is
that the former has memory management stuff all over the place, while
the latter keeps it all together in the <code>memblock</code> class. Normally
STL containers are resized by calling <code>new</code> to create more storage
and then copying the elements there from the old one. This method wastes
space by fragmenting memory, wastes time by copying all the existing data
to the new location, and wastes codespace by having to instantiate all
the resizing code for each and every container type you have. This method
is also absolutely necessary to do this resizing in a perfectly object-safe
way. The uSTL way is to manage memory as an opaque, typeless block, and
then use the container templates to cast it to an appropriate pointer type.
</p><p>
This works just fine, except for one little catch: there is one type
of object you can't store in uSTL containers -- the kind that has pointers
to itself. In other implementations, resizing actually creates new objects
in the new location and destroys them in the old location. uSTL simply
memcpys them there without calling the copy constructor. In other words,
the object can not rely on staying at the same address. Most objects really
don't care. Note that this is not the same thing as doing a bitwise copy,
that you were rightly warned against before! It's a bitwise <em>move</em>
that doesn't create a new object, but simply relocates an existing one.
</p><p>
What this one small concession does is allow aggregation of all memory
management in one place, namely, the <code>memblock</code> class. All the
containers are thus converted mostly into typecasting wrappers that
exist to ensure type safety. Look at the assembly code and you'll see
mostly calls to memblock's functions. This is precisely the feature
that allows reduction in code instantiated by container templates.
</p><p>
However, memblock's usefulness doesn't end there! It can now replace
all your dynamically allocated buffers that you use for unstructured
data. Need to read a file? Don't use new to allocate memory; use a
memblock! It even has a friendly read_file member function for just
that purpose. Need to write a file? Use the write_file call! Unless
you are working with a database or some really large archive, you
should be able to load all your files this way. Imagine, not having
to worry about file I/O again! It's much nicer to work with data in
memory; you know how long it is, so you know when to stop. You can
seek with impunity, and any operations have the cost of a memcpy.
</p><p>
Memblock is derived from memlink, an object for linking to a memory
block. Now you get to store a pointer and the size of whatever it
points to, but with uSTL you can use a memlink object to keep them
together, reducing source clutter and making your code easier to
read and maintain. You can link to constant blocks too with cmemlink,
from which memlink is derived. Because all three are in a single
hierarchy, you never need to care whether you're working on an
allocated block or on somebody else's allocated block. Pointers are
kept together with block sizes, memory is freed when necessary,
and you never have to call new or delete again. Who needs garbage
collection? Memblocks give you the same functionality at a fraction
of the cost.
</p><p>
Linking is not limited to memlink. You can link memblock objects.
You can link string objects. You can even link containers! Now
you can use alloca to create a vector on the stack; use the
<code>typed_alloca_link(v,int,99)</code> macro. All linked objects
will allocate memory and copy the linked data when you increase their
size. You can also do it explicitly by calling <code>copy_link</code>.
Why link? It's cheaper than copying and easier than keeping track
of pointers. For example, here's a line parser:
</p><pre>
string buf, line;
buf.read_file ("some_config_file.txt");
for (auto i = 0u; i &lt; buf.size(); i += line.size() + 1) {
    line.link (buf.iat(i), buf.iat (buf.find ('\n',i)));
    process_line (line);
}
</pre><p>
This way process_line gets a string object instead of a pointer and
a size. If you don't rely on the string being null-terminated, which
basically means not using libc functions on it, this is all you need.
Otherwise buf will have to be writable and you can replace the newline
with a null. In either case you are using no extra heap. The overhead
of link is negligible in most cases, but if you really want to do this
in a tight loop, you can use relink call, which expands completely
inline into one or two instructions, avoiding the virtual unlink() call.
</p>
<h2 id="Streams">Streams</h2>
<p>
The C++ standard library provides global stream objects called cin,
cout, and cerr to replace printf and friends for accessing stdin, stdout,
and stderr, respectively. uSTL versions work mostly the same as the
standard ones (yes, the <code>format</code> call is a uSTL extension). Most
calls use snprintf for output and thus use whatever locale libc uses.
</p><pre>
cout &lt;&lt; "Hello world!" &lt;&lt; endl;
cout &lt;&lt; 456 &lt;&lt; ios::hex &lt;&lt; 0x1234 &lt;&lt; endl;
cerr.format ("You objects are at 0x%08X\n", &amp;o);
</pre><p>
String-writing streams are also available:
</p><pre>
ostringstream os;
os &lt;&lt; "Writing " &lt;&lt; n &lt;&lt; " objects somewhere" &lt;&lt; endl;
cout &lt;&lt; os.str() &lt;&lt; endl;
</pre><p>
fstream is a file access interface with exception handling for errors:
</p><pre>
fstream f;
// C++ standard says that fstream does not throw by default,
f.exceptions (fstream::allbadbits);	// so this enables throwing.
f.open ("file.dat", ios::in| ios::out); // throws file_exception
f.read (buf, bufSize);			// let's read something
f.seek (334455);			// go somewhere
f.write (buf2, buf2Size);		// and write something
f.fnctl (FCNTLID(F_SETFL), O_NONBLOCK); // yup, ALL file operations
memlink l = f.mmap (bufSize, offset);	// even mmap
fill (l, 0);
f.msync (l);
f.munmap (l);
f.close();	// also throws file_exception (with filename!)
</pre><p>
istream and ostream, which are not really usable by themselves in the
standard implementation, are hijacked by uSTL to implement binary data
input and output:
</p><pre>
const size_t writtenSize =
    Align (stream_size_of(number) +
           stream_size_of(ctr)) + 
    stream_size_of(n) +
    stream_size_of(v);
memblock buf (writtenSize);
ostream os (buf);
os &lt;&lt; number &lt;&lt; ctr;
os.align();
os &lt;&lt; n &lt;&lt; v;
</pre><p>
These operations are all very efficient, approaching a straight memcpy
in performance. ostream will not resize the buffer, hence the necessity
to estimate the final size. Most stream_size_of calls are computed at
compile time and thus produce no code. Because the data is written as
is, it is necessary to consider proper data alignment; for example,
a 4 byte int can not be written at stream offset 2. Some architectures
(Macs) actually crash when doing it; Intel processors just do it slowly.
Hence the need to pack the data to a proper "grain". The default align
call will pack to the maximum necessary grain, but can be given an
argument to change that. In case you're wondering, the reason for all
these idiosyncracies is optimization. The smallest and fastest possible
code to dump your stuff into a binary file is produced by this method.
uSTL defines flow operators to write integral values, strings, and
containers, but you can custom-serialize your objects like this:
</p><pre>
/// Some class I want to serialize
class CMyClass {
public:
    void	read (istream&amp; is);
		    { is &gt;&gt; _elements &gt;&gt; _someSize &gt;&gt; _someObject; }
    void	write (ostream&amp; os) const;
		    { os &lt;&lt; _elements &lt;&lt; _someSize &lt;&lt; _someObject; }
    size_t	stream_size (void) const {
		    return (stream_size_of (_elements) +
			    stream_size_of (_someSize) +
			    stream_size_of (_someObject));
		}
private:
    vector&lt;int&gt;	_elements;	///&lt; A bunch of elements.
    size_t	_someSize;	///&lt; Some integral value.
    MyObject	_someObject;	///&lt; Some other streamable object.
}

CMyClass o;
memblock buf (stream_size_of(o));
ostream os (buf);
os &lt;&lt; o;
buf.write_file ("output.dat");
</pre>
<h2 id="Arrays">Arrays</h2>
<p>
One last container I'll mention is <code>array</code>, which is
a fixed-size array of identical elements. There is also the older
version of this called <code>tuple</code>, with template parameters
in reversed order, written for uSTL before C++11 came out with
<code>array</code>. These classes are good for graphical objects like
points, sizes, rectangles, triangles, etc. tuples have SIMD instruction
optimizations, which may or may not be a benefit. Any C-style fixed
size-array works better as one of these, since it becomes a standard
STL container, which you can use with any algorithm, copy by assignment,
initialize in the constructor, etc.
</p><pre>
using coord_t = short int;
using Point2d = tuple&lt;2, coord_t&gt;;
using Size2d = tuple&lt;2, coord_t&gt;;
using Rect = array&lt;Point2d, 2&gt;;

Rect r (Point2d (1,2), Point2d (3,4));
r += Size2d (4, 4);
r[1] -= Size2d (1, 1);
for (auto&amp; i : r)
    TransformPoint (i);
Point2d pt (1, 2);
pt += r[0];
pt *= 2;
</pre>
<h2 id="Exceptions">Exceptions</h2>
<p>
uSTL implements all the standard exception classes defined by the
C++ standard. The exception tree is standalone, but is derived
from std::exception when compiling with libstdc++ for ease of
catching everything. uSTL exceptions implement some additional useful
features. First, they are completely serializable. You can write them
as a binary blob into a file, send them over a network, and handle them
somewhere else. Each exception will print an informative error message
directly to a text stream, reducing your try/catch block to:
</p><pre>
try {
    DoSomething();
} catch (exception&amp; e) {
    cerr &lt;&lt; "Error: " &lt;&lt; e &lt;&lt; endl;
    #ifndef NDEBUG
	cerr &lt;&lt; e.backtrace();
    #endif
} catch (...) {
    cerr &lt;&lt; "Unexpected fatal error has occured.\n";
}
</pre><p>
Second, each exception stores a backtrace (callstack) at the time
of throwing and can print that backtrace as easily as the above
example illustrates. While it is indeed a good practice to design
your exceptions so that you should not care where it was thrown from,
situations occasionally arise while debugging where knowing the thrower
is useful to fix the bug a little faster than otherwise.
</p><p>
Finally, there are additional exception classes for dealing with libc
function errors, file errors, and stream classes. system_error can
be thrown whenever a libc function fails, immediately telling you
what the function call was and the errno description of the failure.
file_exception, thrown by fstream operations, also contains the file name,
which can be pretty darn useful. stream_bounds_exception is extremely
useful in debugging corrupted data, as it tells you exactly where the
corruption starts and what you were trying to read there.
</p>
<h2 id="Contact">Bug reporting</h2>
<p>
Report bugs through the
<a href="https://github.com/msharov/ustl/issues">project issue tracker</a>.
</p>
</div></div>
</body></html>
